What makes a difference in the post-training of LLMs? We investigate the training patterns of different layers in large language models (LLMs), through the lens of gradient, when training with different responses and initial models. We are specifically interested in how fast vs. slow thinking affects the layer-wise gradients, given the recent popularity of training LLMs on reasoning paths such as chain-of-thoughts (CoT) and process rewards. In our study, fast thinking without CoT leads to larger gradients and larger differences of gradients across layers than slow thinking (Detailed CoT), indicating the learning stability brought by the latter. Moreover, pre-trained LLMs are less affected by the instability of fast thinking than instruction-tuned LLMs. Additionally, we study whether the gradient patterns can reflect the correctness of responses when training different LLMs using slow vs. fast thinking paths. The results show that the gradients of slow thinking can distinguish correct and irrelevant reasoning paths. As a comparison, we conduct similar gradient analyses on non-reasoning knowledge learning tasks, on which, however, trivially increasing the response length does not lead to similar behaviors of slow thinking. Our study strengthens fundamental understandings of LLM training and sheds novel insights on its efficiency and stability, which pave the way towards building a generalizable System-2 agent. Our code, data, and gradient statistics can be found in: https://github.com/MingLiiii/Layer_Gradient.

Given a set of known numbers, there are many measures of the degree of inhomogeneity within the set such as the standard deviation, the relative mean difference, and the Gini coefficient. This paper discusses conceptual issues (such as qualitative versus quantitative diversity, and the group as a population versus as a sample), desired properties (such as symmetry and invariance properties), and technical considerations (such as working with differences versus deviations, or absolute versus squared values) in choosing an index suitable for describing the degree of inhomogeneity or diversity in a group of people or computer agents. In particular, it is argued that the relative mean difference and the Gini coefficient are not well-suited as indexes of cultural diversity. This paper then addresses two apparently neglected inverse problems: Given a pre-specified degree of inhomogeneity, what set of unknown numbers has the desired degree of inhomogeneity? And, in particular, what set has the maximal possible degree of inhomogeneity? The solution requires that the set of permissible numbers be bounded with minimum and maximum values. A key benefit of such inverse procedures is that agent-based groups with pre-selected degrees of cultural diversity can be formed to test hypotheses using the full range of possible diversities and thereby avoid statistical problems due to restriction of range effects.

We present an analog VLSI chip for parallel analog vector quantization. TheMOSIS 2.0 J..Lm double-poly CMOS Tiny chip contains an array of 16 x 16 charge-based distance estimation cells, implementing a mean absolute difference (MAD) metric operating on a 16-input analog vector field and 16 analog template vectors.

We present an analog VLSI chip for parallel analog vector quantization. The MOSIS 2.0 J..Lm double-poly CMOS Tiny chip contains an array of 16 x 16 charge-based distance estimation cells, implementing a mean absolute difference (MAD) metric operating on a 16-input analog vector field and 16 analog template vectors.

We present an analog VLSI chip for parallel analog vector quantization. The MOSIS 2.0 J..Lm double-poly CMOS Tiny chip contains an array of 16 x 16 charge-based distance estimation cells, implementing a mean absolute difference (MAD) metric operating on a 16-input analog vector field and 16 analog template vectors.